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I, Andries van Dam, declare: 

1. I am a tenured professor in the Computer Science department of Brown University, 
where I hold the position of Thomas J. Watson, Jr. University Professor of Technology and 
Education Chair and am also a Professor of Computer Science. I have been asked to provide an 
expert declaration on behalf of Samsung Electronics Co. Ltd., Samsung Electronics America, Inc., 
and Samsung Telecommunications America, LLC (collectively “Samsung’’) in the above- 
captioned case. 

om I submit this declaration in support of Samsung’s Opposition to Apple’s Motion for 
a Preliminary Injunction. If asked at hearings or trial, I am prepared to testify regarding the 
matters I discuss in this declaration. 

ce I reserve the right to supplement or amend this declaration based on any new 
information that is relevant to my opinions. 

4. I am being compensated for my work in this matter at the rate of $1000 per hour 
plus expenses. My compensation is in no way tied to the outcome of this matter. 

I. PROFESSIONAL BACKGROUND 

5. A detailed record of my professional qualifications is attached as Exhibit | to this 
declaration and summarized below. 

6. I received a B.S. in Engineering Sciences from Swarthmore College in 1960, and 
an M.S. and Ph.D. in Electrical Engineering from the University of Pennsylvania in 1963 and 
1966 respectively. 

cs I have taught at Brown University since 1965, where I started as an Assistant 
Professor teaching Computer Science in the Division of Applied Mathematics. In 1968 I became 
a tenured Associate Professor of Applied Mathematics, and in 1972 I was promoted to Full 
Professor. In 1976, I became a Professor of Computer Science, and have taught Computer 
Science continuously since 1965. I have held various positions at Brown University, including 
Chairman of the Computer Science Program (1976-1979), Founding Chairman of the Department 
of Computer Science (1979-1985), L. Herbert Ballou University Professor Chair (1992-1995), 


Thomas J. Watson, Jr. University Professor of Technology and Education Chair (1995-present), 
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and Vice President for Research (2002-2006). I have also served as a visiting professor on 
Sabbatical leave to teach and start research groups in Computer Graphics at University of 
Nijmegen in the Netherlands and University of Geneva in Switzerland. 

8. I have also served as the Director of the National Science Foundation Science & 
Technology Center for Computer Graphics and Scientific Visualization (the STC). The STC was 
physically located across 5 universities, including Brown and ran for its allotted 11 years, with its 
financial home at the University of Utah. In my role as director, which I filled for three years, I 
was logistically responsible for the operation and the research programs of the Center. 

9. While on my year’s Sabbatical at the University of Geneva in 1978-79 I was also 
Visiting Scientific Associate at CERN, the European Nuclear Research Institute in Geneva and 
was invited back for many visits to consult and lecture. While at CERN as a Visiting Scientific 
Associate, I co-designed a special-purpose microcomputer specializing in fast event processing for 
handling data from physics experiments, and its microprogramming, and gave various lectures. 
My subsequent visits generally involved consultation on a variety of subjects relating to 
workstations, scientific visualization, and hypermedia. 

10. I have over forty years of experience in the fields of computer graphics, 
hypermedia systems and user interfaces. In my research, I have recently worked on projects 
relating to pen- and touch-centric computing, educational software, and electronic book authoring 
and delivery systems. I have authored or co-authored 120 articles, 9 books, and 3 National 
Research Council Reports. I have presented over 44 invited lectures since 2000. My lectures in 
the past two decades have been primarily focused on the area of interaction in immersive virtual 
environments and scientific visualization, with a recent focus on pen- and touch- computing. I 
have publicly shown work on pen computing on tablet PCs and touch computing on Microsoft 
Surface devices, using both research-based and commercial devices. I have most recently 
focused on applications in digital humanities (or, as it has become known, “e-humanities’”). For 
example, I worked on a humanities project called Large Artwork Displayed on the Surface 
(LADS) for examining large pieces of artwork on any touch-enabled surface supported by 


Windows 7. [also recently helped design a scholarship tool to allow users to easily create 
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selections of hyperlinked multimedia documents, entitled WorkTop. Before we acquired a 
Microsoft Surface, my students had built our own “touch table,” a “home brew” prototype touch 
device, for which we had created multiple applications. My group’s most recent work on touch 
computing has been sponsored by both Microsoft Research and Sharp. I have shown multiple 
unpublished projects using touch computing at the annual Microsoft Faculty Summit. My group 
and I have also produced the Garibaldi Panorama Application, a precursor to LADS, which was 
shown to thousands of people as a key exhibit in a special exhibit at British Library on the future 
of digital scholarship. 

Il. APPLICABLE LEGAL PRINCIPLES 





11. I am informed by counsel that “prior art” includes public information, public 
knowledge, and public acts that occur before an application for a patent was filed. Prior art 
includes patents, journals, Internet publications, systems and products. 

12. I am further informed by counsel that prior art renders a patent claim invalid by 
anticipation if the reference discloses each element of the claim, either expressly or inherently, 
from the point of view of a person of ordinary skill in the art at the time the claimed invention was 
made. I understand that an element of a claim is inherent if it is necessarily present in an item of 
prior art. In my opinion, a person of ordinary skill in the art would have a computer science 
degree, and 3-5 years of software design and implementation experience, including experience 
with graphical user interface design, or would have equivalent educational and work experience. 

LS I am informed by counsel that even if a patent is not anticipated, it is still invalid if 
the differences between the claimed subject matter and the prior art are such that the subject matter 
as a whole would have been obvious at the time the invention was made to a person of ordinary 
skill in the pertinent art. I further understand that a person of ordinary skill is a hypothetical 
person who is presumed to be aware of all the pertinent art. I understand that a person of ordinary 
skill in the art provides a reference point from which the prior art and claimed invention should be 
viewed. This reference point prevents one from using her own insight or hindsight in deciding 


whether a claim is obvious. 
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14. I have been informed that claims directed to a combination of familiar elements 
according to known methods are invalid as obvious when the combination does no more than yield 
predictable results. 

1S. I also understand that practical and common sense considerations should guide a 
proper obviousness analysis, because familiar items may have obvious uses beyond their primary 
purposes. For example, I understand that if a technique has been used to improve one device, and 
a person of ordinary skill in the art would recognize that it would improve similar devices in the 
same way, using the technique is obvious unless its actual application is beyond her skill. 

16. I understand that an obviousness evaluation can be based on a combination of 
multiple prior art references. I understand that prior art references themselves may provide a 
suggestion, motivation, or reason to combine elements of multiple prior art references in the way 
the claimed new invention does. I further understand that the nexus linking two or more prior art 
references or practices may be simple common sense. 

iw I further understand that a claim can be obvious in light of a single reference, 
without the need to combine references, if the elements of the claim that are not found explicitly or 
inherently in the reference can be supplied by the common sense and knowledge of one of skill in 
the art. 

18. I understand that obviousness analysis takes into account the inferences and 
creative steps that a person of ordinary skill in the art would employ under the circumstances, 
because a person of ordinary skill in the art looking to overcome a problem will often be able to fit 
the teachings of multiple publications together like pieces of a puzzle. 

I. MATERIALS CONSIDERED 

19. In forming my opinions in this declaration, I reviewed a number of materials, 
including U.S. Patent No. 7,469,381 (“the ‘381 patent’), its prosecution file history, its 
reexamination file history, and the references cited therein. I also reviewed Dr. Ravin 
Balakrishnan’s Declaration In Support of Apple’s Motion for a Preliminary Injunction, as well as 


the portions of Dr. Balakrishnan’s deposition transcript that I have been permitted to review to 
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date. I have also reviewed the materials cited in this declaration and in the charts attached to this 
declaration as Exhibits 2 - 5. 


IV. OVERVIEW OF THE ‘381 PATENT AND THE ASSERTED CLAIMS 





20. The ‘381 Patent, titled “List Scrolling and Document Translation, Scaling, and 
Rotation on a Touch-Screen Display,” was filed on December 14, 2007 and issued on December 
23, 2008. It claims priority to a number of provisional applications, the earliest of which was 
filed on January 7, 2007. The patent has one named inventor, Bas Ording. 

Al The *381 Patent’s claims relate to a method of translating an electronic document 
displayed on a touch screen display in response to detecting movement of an object on or near the 
touch screen. The ‘381 Patent describes a “snap-back” feature where, if the user translates an 
electronic document beyond the edge of that document, an area beyond that edge will be 
displayed. When the user lifts her finger from the touch screen, the document will “snap” back so 
that the edge of the document is aligned with the edge of the screen, such that no area beyond the 
edge of the document remains in view. 

22. I understand that Apple has asserted that Samsung’s devices infringe claims 1-5, 7, 
9-10, 13-14, 16, 19-20 of the *381 Patent in its Motion for a Preliminary Injunction. Claims 1, 
19, and 20 are independent claims, and the remaining claims are dependent on claim 1. 

V. CLAIM CONSTRUCTION 

23: I understand that the Court has not yet issued a claim construction order in this 
case. In this declaration, I have attempted to apply the claim constructions that Apple and Dr. 
Balakrishnan appear to have applied. My understanding of those claim constructions is based on 
my review of Dr. Ravin Balakrishnan’s Declaration In Support of Apple’s Motion for a 
Preliminary Injunction, as well as the portions of Dr. Balakrishnan’s deposition transcript that I 
have been permitted to review to date, as well as my review of the claim chart Apple filed in 
conjunction with its action before the ITC against HTC Corp., In the Matter of Certain Portable 
Electronic Devices and Related Software, Investigation No. 337-TA-797. 


Vi. PRIOR ART OVERVIEW. 
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24. I understand that Benjamin Bederson and his colleagues created a graphical user 
interface for mobile devices in 2004 known as LaunchTile. I understand that Dr. Bederson and 
his colleagues created a variant of the LaunchTile software, which they entitled XNav. 

25: I have reviewed the declaration of Benjamin Bederson , as well as exhibits to that 
declaration, in forming my opinions in this declaration. I have also personally used a Compaq 
iPaq h1900 series model 1950 PocketPC device running LaunchTile, as well as a Sony VGN- 
U750P tablet device running XNav, in forming my opinions. 

26. I have also provided guidance to attorneys regarding the preparation of screenshots 
appearing in this declaration demonstrating the functionality of the LaunchTile and XNav 
software, and approved the screenshots appearing herein. I likewise guided and approved the 
preparation of videos attached to this declaration as Exhibits 6 - 11. 

27.  L understand that the LaunchTile application was presented at two events in 2005 - 
the ACM Conference on Human Factors in Computing Systems, known as the CHI Conference, in 
April 2005, and a symposium at the Human-Computer Interaction Lab at the University of 
Maryland in May 2005, known as the HCIL Symposium. I further understand that Dr. 
Bederson’s team discussed their work on LaunchTile at these events. I also understand that Dr. 
Bederson’s team gave live demonstrations of a prototype LaunchTile device at the HCIL 
Symposium. [also understand that a device running XNav, as well as a device running 
LaunchTile, was demonstrated in a video presentation that Dr. Bederson made available on his 
web page around April 2005. I further understand that Dr. Bederson provided source code for the 
XNav application to Microsoft in August 2005, and that this source code was made available to 
Microsoft without any restriction on Microsoft’s ability to reproduce, use, or disseminate that 
code. 

28. I have been provided with a copy of XNav source code. I have examined portions 
of this code in forming my opinions herein. 

29. I understand from Dr. Bederson’s declaration and from my own review of the 
XNav software that XNav has identical or substantially similar functionality to the relevant 


LaunchTile functionality described below. In the remainder of my declaration, unless otherwise 


-6- Case No. 11-cv-01846-LHK 


DECLARATION OF ANDRIES VAN DAM, PH.D. INSUPPORT OF SAMSUNG’S OPPOSITION TO 
APPLE’S MOTION FOR A PRELIMINARY INJUNCTION 





NY Dn Oo FR WOW Ye 


oOo ww 


10 
11 
12 
13 
14 
15 
16 
Ly 
18 
19 
20 
21 
2D 
23 
24 
25 
26 
pe | 
28 


Case5:11-cv-01846-LHK Documenti68 Filed08/22/11 Pages of 50 


noted, my discussion and opinions regarding LaunchTile apply equally to the corresponding 
functionality in XNav. 

30. LaunchTile contains a prototype e-mail application. Using this prototype 
application, a user is presented with an inbox comprised of a vertical list of e-mails. LaunchTile 
allows the user to touch the screen and drag a finger or stylus upward or downward, to scroll up or 
down through this e-mail list. When the user reaches the end of the list by scrolling, if the user 
continues to scroll in the same direction, LaunchTile indicates that the user has scrolled beyond 
the edge of the list (“over-scrolled”). It does so by displaying an area of empty space (i.e., an area 
of a solid color) below the last e-mail in the list. If the user’s over-scroll does not exceed a 
threshold distance, after the user ends the scrolling operation by lifting their finger or their stylus 
off the touch screen LaunchTile will perform an animated “elastic snap” to move the e-mail list 
back by scrolling in the opposite direction. As a result of this “snap,” the top or bottom boundary 
of the e-mail list becomes aligned with the boundary of the e-mail display window. Alternatively, 
if the user’s over-scroll continues beyond a threshold distance, the area of empty space will remain 
displayed when the user lifts their finger or stylus off the touch screen. 

Sl. A similar animated “snap” operation is used for LaunchTile’s tile navigation 
system. The LaunchTile application contains a set of 36 tiles, each representing an application. 
The entire set of 36 tiles is called a ‘““World;” the World is divided into nine “Zones” of four tiles 


each. As shown below, the “World” view displays all nine “Zones:” 
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32. A LaunchTile user looking at the World can view a Zone by pressing on a blue 
button in the center of the Zone. The user can then swipe her finger to scroll to a neighboring 
Zone. The user may lift her finger while performing the swiping gesture; when this happens, if 
the user has not swiped beyond a threshold distance in the direction of the next Zone, the current 
Zone will “snap” back into its previous position, aligning the edges of the Zone with the edges of 
the screen. See below paragraphs 39 - 68. 

33. In addition to the LaunchTile and XNav systems, a number of prior art publications 
disclose techniques recited in the ‘381 patent’s claims. For example, WO 03/081458 to Lira (and 
US. Patent No. 7,872,640 to Lira) disclose methods for browsing a large electronic document, 
such as a web page or column of a web page, using a small touch-screen display, by scrolling 
through the page using their finger or a stylus. When the user lifts their finger or stylus off the 
touch-screen display, Lira will “snap” the browsing window to the edge of the nearest column of 
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the web page. Lira further discloses that this “snap” feature “can also be animated to provide an 
appearance of movement as the display scrolls.” (/d. at 15.) The Lira reference is discussed in 
greater detail in Exhibit 4. 

34. Likewise, U.S. Patent No. 7,152,210 to Van Den Hoven (from PCT/EP00/10286) 
discloses scrolling operations for electronic documents (digital images) in response to touch- 
screen input. These operations include modifying scrolling in response to a user removing their 
finger from a touch-screen display, as well as operations for reversing the direction of translation. 
The Van Den Hoven reference is discussed in greater detail in Exhibit 5. 

Vil. INVALIDITY 

35: I have compared the LaunchTile and XNav applications with the claims of the ‘381 
Patent. In my opinion, LaunchTile and XNav each disclose each and every limitation of the 
asserted claims of the ‘381 patent. In addition, in the event that either LaunchTile or XNav do 
not disclose each and every limitation of the asserted claims of the ‘381 patent, in my opinion the 
claims would be obvious in light of either of these two systems.. 

36. Attached hereto as Exhibits 2 and 3 are detailed charts comparing the LaunchTile 
and XNav systems to the claims of the “381 patent. These charts support and form the basis for 
my opinions, and I incorporate those charts herein by reference. 

Shs Moreover, in my opinion the Lira and Van Den Hoven references each disclose 
each and every limitation of the asserted claims of the ‘381 patent and/or render those claims 
obvious. Attached hereto as Exhibits 4 and 5 are detailed charts comparing the Lira and Van Den 
Hoven references to the claims of the ‘381 patent. These charts also support and form the basis 
for my opinions, and I incorporate those charts herein by reference. 

38. Below I discuss particular disclosures relating to LaunchTile. As explained above, 
XNav has identical or substantially similar functionality to the LaunchTile functionality described 
below, and my discussion of and opinions regarding LaunchTile below apply equally to 
corresponding functionality in XNav. 

A. Claim 1 


1. Preamble 
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39. The Preamble recites ““A computer-implemented method.” A mobile computing 
device running the LaunchTile application includes a computer-implemented method. For 
example, I ran the LaunchTile program on a Compaq iPaq h1900 series model 1950 PocketPC’, 
which is a mobile computing device that includes a Samsung SC32442 processor and main 
memory of 32 MB SDRAM. See Ex. 12 at B-1 (Compaq iPaq h1900 Series Manual); Ex. 13 at p. 
3 (HP iPaq 1950 Pocket PC QuickSpecs). 

2 Element 1 

40. Element | of claim 1| recites: “[method] comprising: at a device with a touch screen 
display.” 

41. The Compaq iPaq h1900 series model 1950 PocketPC has a touchscreen display. 
For example, this device responds to touches from a stylus or a finger. LaunchTile runs on the 
Compaq iPaq h1900 series model 1950 PocketPC device and other similar devices with touch 


screen displays. 





It is my understanding that the LaunchTile application can also run on any number of 
Windows Mobile devices other than the Compaq iPaq h1900 series PocketPC as well. As noted 
above, I have personally used LaunchTile on a Compaq iPaq h1900 series model 1950 PocketPC 
device. My opinions with regard to a Compaq iPaq h1900 series model 1950 PocketPC running 
LaunchTile apply equally to any other touch screen device that is capable of running the 
LaunchTile software. 
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42. It is my opinion that LaunchTile running on a Compaq iPaq h1900 series model 

1950 PocketPC discloses this element of Claim 1. 
2: Element 2 

43. Element 2 of claim | recites: “displaying a first portion of an electronic document.” 

44. —_ LaunchTile displays a first portion of an electronic document. I discuss several 
representative examples below. 

45. In my opinion, a person of ordinary skill in the art would understand that electronic 
documents, like their real-life counterparts, can be arranged in a hierarchy. Accordingly, the 
electronic document recited in claim | may itself contain sub-documents. Likewise, the 
electronic document recited in claim | may itself be a sub-document within the hierarchy of some 
other electronic document. This concept is familiar from the real world, in which a document 
such as a book, binder, or legal pleading may contain many sub-documents, such as chapters, 
individual pages, or exhibits. Likewise, a document such as a page or an exhibit may itself be 
part of a larger document. 

46. My opinion is supported by my review of a claim chart Apple filed in conjunction 
with its complaint before the ITC against HTC Corp., In the Matter of Certain Portable Electronic 
Devices and Related Software, Investigation No. 337-TA-797. In Apple’s claim chart for the 
‘381 patent (Ex. 13 to its Complaint), Apple has asserted that the electronic document recited in 
claim | includes an electronic document comprised of multiple individual sub-documents, such as 
contiguous digital images. 

47. My opinion is further supported by Dr. Balakrishnan’s deposition testimony. In 
that testimony, Dr. Balakrisnhan defined an electronic document in the context of the *381 patent 
claims as something that can be visually representable on a screen and that has a defined set of 
boundaries, such as (but not limited to) length and width: 


Q. Is the electronic doc -- what -- what does an "electronic document" mean in the context 
of this patent? 


MR. MONACH: Object to form to the extent it's calling for a legal conclusion, but you can 
give your views on that. 
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THE WITNESS: In the context of this patent, my understanding, having read the patent 
and the claims, is the electronic document is some visual representation on the screen that 
has a defined length and a width, as an example, or defined set of boundaries, because they 
may not have to be a rectangular set of boundaries. 


MR. JOHNSON: Q. So can it — it can be anything with a defined length and width? 


A It could be any visually represented thing with a defined boundary. I'd rather use 
the word "boundaries," because length and width may connote a rectangular thing. It may 
not be a rectangle, necessarily. 


Q Okay. So an electronic document is anything that can be visually represented with a 
defined boundary? 


MR. MONACH: Object to the form of the question; object as calling for a legal 
conclusion. 


THE WITNESS: In the context of this patent and the claims, reading the patent and the 
claims, I would say that would be a -- my definition of an electronic document would be 
something visually representable on the screen that -- that has a defined set of boundaries. 


Balakrishnan Dep. Tr. at 27:13-28:18. Dr. Balakrishnan also testified that within a six-by-six grid 
of thirty-six rectangular shapes, each with a visual boundary, a hierarchy of documents may be 
present: an electronic document could be “one of these, let me call it sub rectangles,” and further 
“could be some combination of them,” depending “on how, you ,now, you want to put the 
boundary around it.” Jd. at 152:10-23. This combination of “sub rectangles” forming an 
electronic document could include, for example, a two-by-four grid of sub-rectangles as one 
electronic document (id. at 152:24-153:19, referencing Ex. 104). Dr. Balakrishnan further 
explained that al electronic document does not need to be comprised of a hierarchical structure 
such as a grid with sub-documents: “I think the electronic doesn’t have to be anything to do with 
the grid. It — it’s any visual thing with defined boundaries — by my definition of it.” Id. at 155:2- 
9 (objections and interruptions omitted). Dr. Balakrishnan also testified that a “portion” of an 
electronic document may be any area of that document, including the entirety of the document; for 
example, an entire digital image can constitute one “portion” of that image. Jd. at 146:15-19. 

48. Representative Example #1. In one example, LaunchTile includes an email 


program capable of displaying a list of email messages, such as an inbox, as an electronic 
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1 || document. The computing device will display a portion of the email list as a first portion, shown 


2 || below in green outline: 
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49. Representative Example #2. In another example, the electronic document is a 
Zone in LaunchTile. As described above a Zone is a collection of four individual tiles. I 
understand that these tiles are typically individual image files (.png files). Each tile is itself an 
electronic document, and a Zone comprised of four adjacent tiles is also an electronic document. 
(As described below in Representative Example #3, in my opinion other configurations of 
adjacent tiles are also electronic documents; for example, eight tiles, or two Zones, can constitute 
the electronic document recited in claim 1.) My opinion is supported by Apple’s apparent 
construction of this term in its infringement contentions in other litigation involving the ‘381 
patent (Ex. 13 to its Complaint against HTC, discussed above), where Apple has identified an 
“electronic document” comprised of multiple contiguous digital images. This is consistent with 
the ordinary meaning of document in the electronic sense or in the real world. In this example, 
the electronic document is the Zone located in the center of the World view. This Zone is 


outlined in green in the picture below: 
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Electronic document 





50. | When the user enters Zone view (e.g., from the World view or from the Application 
Tile view), the user will see a single Zone. The user can scroll to a neighboring Zone by touching 
the touch screen device with a finger or a stylus and moving the finger or stylus. If the user 
begins in the center Zone and moves to the left, a first portion of the Zone will be displayed. The 


first portion is depicted in the picture below, outlined in green: 


First portion 





51. As shown above, when the user scrolls to the left, the four tiles comprising the 
Zone also move left. The dark blue button at the center of the screen, which is an interactive 
button, is not an element of any one of these four tiles or the Zone they comprise. The button is 
instead a control element of the user interface. These elements are collectively referred to as 
“chrome,” “decoration,” or “window dressing” in the field. The blue button, which is one 
“chrome” element of the overall LaunchTile user interface’s “chrome,” is a zoom control button 


that operates as an alternative to other classic zoom control elements, such as sliders or a graphical 
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magnifying glass. It thus does not scroll with the Zone; it remains centered while the Zone 
moves, because it is an area that is effectively superimposed over the Zone in two-dimensional 
space. It is typical for some of the “chrome” elements of a user interface, such as pop-up menus 
and dialogue boxes, to be displayed as effectively superimposed over content. Graphical user 
interfaces are often comprised of several composite layers, which are displayed according to 
different “z-ordering” values assigned to different layers for purposes of window management. 

52. Representative Example #3. As discussed above, any set of contiguous tiles 
could be considered an electronic document. Accordingly, two contiguous Zones comprised of 
eight tiles would also constitute the electronic document recited in claim 1. 

5 As noted above, my opinion is supported by the ordinary meaning of document in 
the electronic and real-world sense, as well as by Apple’s apparent construction of this term in 
parallel litigation. My opinion is also based on my application of Dr. Balakrishnan’s own 
construction of the claim terms. Dr. Balakrishnan testified that the “electronic document” recited 
in the “381 patent’s claims “could be some combination of” sub-documents, depending “on how, 
you know, you want to put the boundary around it.” Balakrishnan Dep. at 152:10-23. This 
arrangement could include, for example, a two-by-four grid of sub-documents as one electronic 
document (id. at 152:24-153:19, referencing Ex. 104). 


54. Two contiguous Zones comprised of eight tiles are shown outlined in green below: 


Electronic document 
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SS. In this example, a first portion of the electronic document is shown in the figure 
below. When the user enters Zone view (e.g., from the World view or from the Application Tile 
view), the user will see four tiles that constitutes a first portion of an electronic document 
comprised of eight tiles. Thus, the four tiles displayed in the figure below are one portion of an 


eight-tile, two-Zone document. 


Electronic document First portion 





4. Element 3 


56. Element 3 of claim | recites: “detecting a movement of an object on or near the 
touch screen display; in response to detecting the movement, translating the electronic document 
displayed on the touch screen display in a first direction to display a second portion of the 
electronic document, wherein the second portion is different from the first portion.” 

57. Representative Example #1. In one example, a Compaq iPaq h1900 series model 
1950 PocketPC running LaunchTile will detect movement of an object, including either a finger or 
a stylus, on the touch screen display of the device on which it is running. In response to that 
detected movement, the application will scroll the list up so that a second portion of the email list 


is displayed: 
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First portion Second portion 





58. Representative Example #2. In another example, in response to that detected 
movement, the application will scroll the displayed Zone in the same direction as the movement. 
For context, as shown below, the “first portion” in Representative Example #2 was obtained by 


having previously scrolled slightly to the left: 


First portion 





Continuing from this position, if the user swipes her finger to the right, the electronic document 


will move to the right. A second portion of the electronic document is then displayed. This 
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second portion of the document is different from the first portion of the document. This is 


depicted in the figures below: 


First portion 


Second portion 





59. Representative Example #3. In another example, where two contiguous Zones 
form a single electronic document, a second portion of that document is displayed when the user 
scrolls. For example, if the user scrolls left, a second Zone comprising the second half of the 
eight-tile electronic document will be displayed. In the sequence below, the first image illustrates 
the starting point just prior to a scrolling operation, and the second and third images represent 


partial movement during the scrolling operation: 


First portion Second portion 





In the representative examples discussing a first or second “direction”, the direction in 
which the LaunchTile device is scrolled is representative — not exhaustive. My opinions are of 
course not limited to the specific scrolling directions used in these screenshots. 
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5. Element 4 

60. Element 4 of claim 1 recites: “in response to an edge of the electronic document 
being reached while translating the electronic document in the first direction while the object is 
still detected on or near the touch screen display: displaying an area beyond the edge of the 
document, and displaying a third portion of the electronic document, wherein the third portion is 
smaller than the first portion.” 

61. Representative Example #1. In one example, in the email program, when the 
user reaches the bottom of the email list when scrolling up, the edge of the email list is displayed. 
Beyond that edge a blank area is also displayed. This blank area is not part of the electronic 
document that consists of the email list. This results in the display of a third portion of the 


document, which is smaller than the first portion of the document, because of the blank area. 


First portion Second portion Third portion 
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Third portion 





62. Representative Example #2. In another example, as the user reaches the left edge 





of a Zone while scrolling to the right in the example discussed above, an area beyond the left edge 
of that Zone will be displayed. That area will consist of a solid-colored “gutter,” or border, 
followed by the neighboring Zone on the left. As the user continues scrolling to the right, a third 
portion of the electronic document is displayed that is smaller than the first portion of the 
electronic document because more of the electronic document has been scrolled off the screen. 
The third portion and the area beyond the edge of the electronic document are displayed in the 


picture below: 
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First portion Second portion Third portion 








IN TetCelalereya tare] Gray “gutter” 


zone 


Area beyond 
the edge 





63. Representative Example #3. In another example, two contiguous Zones 


comprise the electronic document. These Zones are shown below: 
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Electronic document 





In this example, as explained above, where two contiguous Zones form a single electronic 
document, a second portion of the electronic document is now displayed as the user scrolls to the 
left. If the user continues scrolling to the left, the right edge of the electronic document will be 
encountered. As the user scrolls past the edge, an area beyond that edge will be displayed. That 
area will consist of a gray “gutter,” or border, followed by the neighboring Zone, similar to that in 


representative example #2. 


First portion 
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Second portion Third portion 





6. Element 5 

64. Element 5 of claim | recites: “in response to detecting that the object is no longer 
on or near the touch screen display, translating the electronic document in a second direction until 
the area beyond the edge of the electronic document is no longer displayed to display a fourth 
portion of the electronic document, wherein the fourth portion is different from the first portion.” 

65. Representative Example #1. In one example, in LaunchTile in the email 
program, if the user lifts her finger after displaying the third portion of the email list, the email list 
will reverse direction and “snap” back to the bottom edge of the email list, subject to the threshold 
distance limitation discussed above, such that the area beyond this edge will no longer be 
displayed. The result will display a fourth portion of the Zone that is different from the first 


portion. 
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First portion Second portion 





66. Representative Example #2. Similarly, in another example, if the user lifts her 
finger while in the process of scrolling between Zones, the interface will automatically select a 
Zone to snap to. If the user has not scrolled past a particular threshold in the direction of the next 
Zone, the application will snap back to the previous Zone. This snap-back feature will cause the 
electronic document to be translated in a second direction, which will be opposite to the first 
direction, until the area beyond the edge of the electronic document can no longer be seen. Asa 
result, a fourth portion of the document is displayed. I note that Dr. Balakrishnan has advocated a 
claim construction wherein a “portion” of a document can be any area of a document, including 
the entire area of the document. Balakrishnan Dep. Tr. at 146:15-19. 

67. | The fourth portion of the document is different from the first portion. The 
sequence below illustrates this: 
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First portion Second portion 





68. Representative Example #3. In another example, in the case of two contiguous 
Zones being the electronic document, if the user lifts her finger after scrolling some distance 
beyond the second portion of the electronic document, the electronic document will snap back, 
translating in the opposite direction until the area beyond the edge of the electronic document is no 
longer in view. The result will display a fourth portion of the electronic document that is 


different from the first portion. 
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First portion Second portion 








69. Based on this analysis, it is my opinion that a touch screen device running 
LaunchTile discloses each and every element of Claim 1. 

B. Claim 2 

70. Claim 2 covers: “The computer-implemented method of claim 1, wherein the first 
portion of the electronic document, the second portion of the electronic document, the third 
portion of the electronic document, and the fourth portion of the electronic document are displayed 
at the same magnification.” 

71. This claim depends on Claim 1, but adds the additional requirement that all four 
portions of the electronic document be displayed at the same magnification. 

72. As I explained above, a touch screen device running LaunchTile anticipates Claim 


1 of the ‘381 Patent. Additionally, as the images in the previous section indicate, all four portions 
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of the electronic document are displayed at the same magnification. Thus, it is my opinion that a 
touch screen device running LaunchTile discloses each and every element of Claim 2. 

C. Claim 3 

12 Claim 3 covers: “The computer-implemented method of claim 1, wherein the 
movement of the object is on the touch screen display.” 

74. Claim 3, which is dependent on Claim 1, includes the additional requirement that 
the movement of the object is on the touch screen display. For the touch screen device on which 
LaunchTile executes, the movement of the finger or stylus will only be detected when they are 
actually on the touch screen display. Thus, it is my opinion that a touch screen device running 
LaunchTile discloses each and every element of Claim 3. 

D. — Claim 4 

i: Claim 4 covers: “The computer-implemented method of claim 1, wherein the 
object is a finger.” 

76. Claim 4, which is dependent on Claim 1, adds the requirement that the object used 
to scroll the document is a finger. For example, using a Compaq iPaq h1900 series model 1950 
PocketPC to run LaunchTile, a finger can be used to scroll, including to scroll through Zones. 
Thus, it is my opinion that a touch screen device running LaunchTile discloses each and every 
element of Claim 4. 

E. Claim 5 

re Claim 5 covers: “The computer-implemented method of claim 1, wherein the first 
direction is a vertical direction, a horizontal direction, or a diagonal direction.” 

78. The LaunchTile email application discussed above involved vertical movement in a 
first direction. The scrolling operations between Zones discussed above involved horizontal 
movement in a first direction. Thus, it is my opinion that a touch screen device running 
LaunchTile discloses each and every element of Claim 5. 

P. Claim 7 

79. Claim 7 covers: “The computer-implemented method of claim 1, wherein the 


electronic document is a digital image.” 
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80. Claim 7, which is dependent on Claim 1, adds the requirement that the electronic 
document is “a” digital image. I am informed by counsel that in patent claims, the word “a” is 
presumed to mean “one or more.” 

81. A computing device running LaunchTile discloses an electronic document that 
comprises one or more digital images. 

82. Representative Example #1. For example, the LaunchTile prototype email 
application has an electronic document that is at least one or more digital images. In the 
prototype email application, the electronic document is the email list. However, because the 
email application in only a prototype, each entry in the email list is actually an individual digital 
image or .png file, acting as placeholders in order to emulate an operative email application. 

Thus, as the email application is currently written in LaunchTile, the electronic document (email 
list) is composed of one or more digital images (.png files representing entries in the list). 

83. Representative Examples #2 & #3. Similarly, a Zone in LaunchTile is 
comprised of four tiles, and two contiguous Zones are comprised of eight tiles. Each of these tiles 
is displayed as an individual image, and in fact many are .png image files. 

84. To the extent the examples discussed above do not anticipate claim 7, it would have 
been obvious to one of ordinary skill in the art to modify LaunchTile so that the electronic 
document is a single digital image. The user interface features identified above as anticipating 
claim 1 of the *381 patent operate independently of the nature of the underlying electronic 
document. An implementer could employ these same techniques to manipulate any document 
displayed on the screen, regardless of whether the document is a list of items or a digital image. 
Applying these techniques to a digital image would not have presented any unique problems, or 
even significant effort. Additionally, because digital images are commonly displayed on 
computing devices, one of skill in the art would be motivated to modify LaunchTile so that its user 
interface features would operate in the same manner described above when the electronic 
document is a single digital image. Thus, it is my opinion that Claim 7 recites an obvious 
modification to the functionality contained in LaunchTile and in XNav if LaunchTile or XNav do 


not disclose each and every limitation of Claim 7. 
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G. Claim 9 

85. Claim 9 covers: “The computer-implemented method of claim 1, wherein the 
electronic document includes a list of items.” 

86. Representative Example #1. The electronic document in the LaunchTile email 
application includes a list of items. For example, as shown in the image below, the electronic 


document is an inbox comprising a list of emails. 


2 chet store % to 
aj Domke, Judah Save the datet 





87. Representative Examples #2 & #3. Similarly, a Zone (viewed on its own, or 
viewed as part of an eight-tile area of two contiguous Zones) also includes a list of phone-related 
events, including missed calls, outgoing calls, voicemail, and incoming text messages. The 
electronic document shown below (which is half of the electronic document in Representative 


Example #3) also includes a “Calendar,” which consists of a list of scheduled events. 


-29- Case No. 11-cv-01846-LHK 


DECLARATION OF ANDRIES VAN DAM, PH.D. INSUPPORT OF SAMSUNG’S OPPOSITION TO 
APPLE’S MOTION FOR A PRELIMINARY INJUNCTION 





YA Dn Oo FBP WO YPN 


Oo ow 


10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
pe | 
28 


Case5:11-cv-01846-LHK Documenti68 Filed08/22/11 Page31 of 50 


Electronic document 





88. Thus, it is my opinion that a touch screen device running LaunchTile discloses each 
and every element of Claim 9. 

Fl; Claim 10 

89. Claim 10 covers: “The computer-implemented method of claim 1, wherein the 
second direction is opposite the first direction.” 

90. | When the snap-back effect discussed above occurs after the user lifts her finger, the 
direction of the snap-back (the second direction) is opposite the direction in which the user was 
scrolling (the first direction). Thus, it is my opinion that a touch screen device running 
LaunchTile discloses each and every element of Claim 10. 

I. Claim 13 

ol, Claim 13 covers: “The computer-implemented method of claim 1, wherein the area 
beyond the edge of the document is black, gray, a solid color, or white.” 

92. Representative Example #1. In one example, in the email program, when the end 


of the list of emails is reached, the area beyond the edge of the email list is a solid color: 
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93. Representative Examples #2 & #3. In another example, between each of the 





Zones displayed in LaunchTile is a solid-colored gray “gutter” area that indicates a separation 
between the Zones. This gutter area is displayed beyond the edge of the document. 
Accordingly, the gutter area beyond the edge of the document is a solid-colored gray area. This 
same area surrounds an eight-tile area of two contiguous Zones and forms a solid-colored gray 


area beyond the edge of that document. 
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94. I note that this solid-colored gray “gutter” area is the first area displayed beyond the 
edge of the document during the scrolling operation discussed above. Thus, at some point during 
the scrolling operation, the solid-colored gray area is the only area displayed beyond the edge of 
the document. The neighboring Zone area beyond this “gutter” will only displayed later, if the 
user continues to scroll. There is therefore a point in this scrolling operation at which the entire 
area beyond the edge of the document is displayed as a solid-colored gray area. There is further a 
point in this scrolling operation where the third portion of the document would, at a pixel level, be 
smaller than a previously-displayed first portion of the document. 

0; For these reasons, it is my opinion that a touch screen device running LaunchTile 
discloses each and every element of Claim 13. 

96. —_In the event that this limitation was not disclosed explicitly, in my opinion it would 
be obvious to modify LaunchTile, or any other system, such that the area beyond the edge of the 
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document is black, gray, a solid color, or white. This claim recites an issue of color selection. 
Any competent user interface designer understands that solid colors can be, and very frequently 
are, used as elements of a display. Solid colors, including black, gray, or white, are common 
background colors in user interfaces. Their use in this context would be trivial and would involve 
no special effort, skill, or insight. 

J. Claim 14 

OT: Claim 14 covers: “The computer-implemented method of claim 1, wherein the area 
beyond the edge of the document is visually distinct from the document.” 

98. Representative Example #1. In one example, in the email program, the area 
beyond the edge of the email list is visually distinct from the document, i.e. the area beyond the 
edge is a solid color, whereas the email list contains text and images. 

99. Representative Examples #2 & #3. As discussed above, each of the Zones 
displayed in LaunchTile is separated by a solid-colored gray area that indicates a border between 
Zones. This gray area is displayed beyond the edge of the Zone and is visually distinct from the 
Zone. This area is also displayed as a border of an eight-tile area comprised of two contiguous 
Zones. This gray area is displayed beyond the edge of such a document and is visually distinct 
from the document. 

100. Thus, it is my opinion that a touch screen device running LaunchTile discloses each 
and every limitation of Claim 14. 

K. Claim 16 

101. Claim 16 covers: “The computer-implemented method of claim 1, wherein 
changing from translating in the first direction to translating in the second direction until the area 
beyond the edge of the document is no longer displayed makes the edge of the document appear to 
be elastically attached to an edge of the touch screen display or to an edge displayed on the touch 
screen display.” 

102. Representative Example #1. When the email list “snaps” back to align itself with 
the bottom edge of the display window, it makes the edge of the document appear to be elastically 


attached to the bottom edge of the display window. 
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103. Representative Examples #2 & #3. Similarly, when the Zones in LaunchTile 
snap back to align themselves with the edge of the screen, this snap-back effect is animated in a 
way that makes a Zone appear to be elastically attached to the edge of the display. Thus, it is my 
opinion that a touch screen device running LaunchTile discloses each and every limitation of 
Claim 16. 

Le Claim 19 

1. Preamble 

104. To the extent the preamble is a limitation, the LaunchTile application runs on a 
device. In particular, as discussed above, I ran the LaunchTile program on a Compaq iPaq h1900 
series model 1950 PocketPC. 

2s Element | 

105. Element 1 of claim 19 recites: “a touch screen display.” 

106. The Compaq iPaq h1900 series model 1950 PocketPC is a device with a touch 
screen display, as discussed above. Thus it is my opinion that such a device meets this element of 
claim 19. 

a Element 2 

107. Element 2 of claim 19 recites: “one or more processors.” 

108. The Compaq iPaq h1900 series model 1950 PocketPC is a device that has an 
Samsung SC32442 processor. See Ex. 14 at p.3 (HP iPaq 1950 Pocket PC QuickSpecs). Thus, 
it is my opinion that such a device meets this element of claim 19. 

4, Element 3 

109. Element 3 of claim 19 recites: “memory.” 

110. The Compaq iPaq h1900 series model 1950 PocketPC is a device that has 32 MB 
of volatile main memory. See Ex. 14 at p.3 (HP iPaq 1950 Pocket PC QuickSpecs). Thus, it is 
my opinion that such a device meets this element of claim 19. 


5. Element 4 
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111. Element 4 of claim 19 recites: “one or more programs, wherein the one or more 
programs are stored in memory and configured to be executed by the one or more processors, the 
programs including.” 

112. A Compaq iPaq h1900 series model 1950 PocketPC with the LaunchTile 
application installed on it will, when used, have a program stored in memory and configured to be 
executed by the one or more processors. The LaunchTile application can be executed by 
navigating to the LaunchTile application in the File Explorer and clicking on the file. Thus, it is 
my opinion that a Compaq iPaq h1900 series model 1950 PocketPC with the LaunchTile 
application installed on it meets this element of Claim 19. 

6. Element 5 

113. Element 5 of claim 19 recites: “instructions for displaying a first portion of an 
electronic document.” 

114. As described above in Section VI.a.3, the LaunchTile application displays a first 
portion of an electronic document. Because it performs this function, the LaunchTile application 
must also contain instructions for displaying a first portion of an electronic document. Thus, it is 
my opinion that a Compaq iPaq h1900 series model 1950 PocketPC with the LaunchTile 
application installed on it meets this element of Claim 19. 

ve Element 6 

115. Element 6 of claim 19 recites: “instructions for detecting a movement of an object 
on or near the touch screen display; instructions for translating the electronic document displayed 
on the touch screen display in a first direction to display a second portion of the electronic 
document, wherein the second portion is different from the first portion, in response to detecting 
the movement.” 

116. As described above in Section VI.a.4, the LaunchTile application detects a 
movement of an object on or near the touch screen display; in response to detecting the movement, 
translates the electronic document displayed on the touch screen display in a first direction to 
display a second portion of the electronic document, wherein the second portion is different from 


the first portion. Because it performs this function, the LaunchTile application must also contain 
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instructions for doing so. Thus, it is my opinion that a Compaq iPaq h1900 series model 1950 
PocketPC with the LaunchTile application installed on it meets this element of Claim 19. 
8. Element 7 

117. Element 7 of claim 19 recites: “instructions for displaying an area beyond an edge 
of the electronic document and displaying a third portion of the electronic document, wherein the 
third portion is smaller than the first portion, in response to the edge of the electronic document 
being reached while translating the electronic document in the first direction while the object is 
still detected on or near the touch screen display.” 

118. As described above in Section VI.a.5, the LaunchTile application displays an area 
beyond the edge of the electronic document (e.g., a grey or multi-colored area) and displays a third 
portion of the electronic document, wherein the third portion is smaller than the first portion, in 
response to the edge of the electronic document being reached while translating it in the first 
direction while the object is still detected on or near the touch screen display. Because it 
performs this function, the LaunchTile application must also contain instructions for doing so. 
Thus, it is my opinion that a Compaq iPaq h1900 series model 1950 PocketPC with the 
LaunchTile application installed on it meets this element of Claim 19. 

9. Element 8 

119. Element 8 of claim 19 recites: “instructions for translating the electronic document 
in a second direction until the area beyond the edge of the electronic document is no longer 
displayed to display a fourth portion of the electronic document, wherein the fourth portion is 
different from the first portion, in response to detecting that the object is no longer on or near the 
touch screen display.” 

120. As described above in Section VI.a.6, in response to detecting that the object is no 
longer on or near the touch screen display, the LaunchTile application translates the electronic 
document in a second direction until the area beyond the edge of the electronic document is no 
longer displayed to display a fourth portion of the electronic document that is different from the 


first portion. Because it performs this function, the LaunchTile application must also contain 
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instructions for doing so. Thus, it is my opinion that a Compaq iPaq h1900 series model 1950 
PocketPC with the LaunchTile application installed on it meets this element of Claim 19. 

121. In my opinion, a Compaq iPaq h1900 series model 1950 PocketPC with the 
LaunchTile application installed on it discloses each and every limitation of Claim 19. 

M. Claim 20 

1. Preamble 

122. The preamble of claim 20 recites: ““A computer readable storage medium having 
stored therein instructions, which when executed by a device with a touch screen display, cause a 
device to.” 

123. A Compaq iPaq h1900 series model 1950 PocketPC device with the LaunchTile 
application installed on it is device containing a computer readable storage medium with 
instructions for the LaunchTile application stored therein that cause it to perform the functions 
discussed below. Furthermore, this device includes a touch screen display. 

2: Element | 

124. Element 1 of claim 20 recites: “display a first portion of an electronic document” 

125. As described above, the LaunchTile application displays a first portion of an 
electronic document. Because it performs this function, the LaunchTile application must also 
contain instructions for displaying a first portion of an electronic document and those instructions 
must be stored in a computer readable storage medium. Thus, it is my opinion that a Compaq 
iPaq h1900 series model 1950 PocketPC with the LaunchTile application installed on it meets this 
element of Claim 20. 

3. Element 2 

126. Element 2 of claim 20 recites: “detect a movement of an object on or near the touch 
screen display; translate the electronic document displayed on the touch screen display in a first 
direction to display a second portion of the electronic document, wherein the second portion 1s 
different from the first portion, in response to detecting the movement.” 

127. As described above in Section VI.a.4, the LaunchTile application detects a 


movement of an object on or near the touch screen display; in response to detecting the movement, 
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translates the electronic document displayed on the touch screen display in a first direction to 
display a second portion of the electronic document, wherein the second portion is different from 
the first portion. Because it performs this function, the LaunchTile application must also contain 
instructions for doing so and those instructions must be stored in a computer readable storage 
medium. Thus, it is my opinion that a Compaq iPag h1900 series model 1950 PocketPC with the 
LaunchTile application installed on it meets this element of Claim 20. 

4. Element 3 

128. Element 3 of claim 20 recites: “display an area beyond an edge of the electronic 
document and display a third portion of the electronic document, wherein the third portion is 
smaller than the first portion, if the edge of the electronic document is reached while translating 
the electronic document in the first direction while the object is still detected on or near the touch 
screen display.” 

129. As described above in Section VI.a.5, the LaunchTile application displays an area 
beyond the edge of the electronic document (e.g., a grey or multi-colored area) and displays a third 
portion of the electronic document, wherein the third portion is smaller than the first portion, in 
response to the edge of the electronic document being reached while translating it in the first 
direction while the object is still detected on or near the touch screen display. Because it 
performs this function, the LaunchTile application must also contain instructions for doing so and 
those instructions must be stored in a computer readable storage medium. Thus, it is my opinion 
that a Compaq iPaq h1900 series model 1950 PocketPC with the LaunchTile application installed 
on it meets this element of Claim 20. 

a. Element 4 

130. Element 4 of claim 20 recites: “translate the electronic document in a second 
direction until the area beyond the edge of the electronic document is no longer displayed to 
display a fourth portion of the electronic document, wherein the fourth portion is different from 
the first portion, in response to detecting that the object is no longer on or near the touch screen 


display.” 
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131. As described above in Section VI.a.6, in response to detecting that the object is no 
longer on or near the touch screen display, the LaunchTile application translates the electronic 
document in a second direction until the area beyond the edge of the electronic document is no 
longer displayed to display a fourth portion of the electronic document that is different from the 
first portion. Because it performs this function, the LaunchTile application must also contain 
instructions for doing so and those instructions must be stored in a computer readable storage 
medium. Thus, it is my opinion that a Compaq iPaq h1900 series model 1950 PocketPC with the 
LaunchTile application installed on it meets this element of Claim 20. 

132. Accordingly, it is my opinion that a Compaq iPaq h1900 series model 1950 
PocketPC with the LaunchTile application installed on it discloses each and every limitation of 
Claim 20. 

VIN. OBVIOUSNESS CONSIDERATIONS 

133. Iunderstand from counsel that the combination of familiar elements according to 
known methods is likely to be obvious when it does no more than yield predictable results. I 
likewise understand from counsel that if a technique is known to have been used to improve on a 
particular device, and a person of ordinary skill would recognize that this same technique would 
improve similar devices in the same way, using the technique is obvious, unless its actual 
application and implementation is outside the abilities of a person of ordinary skill in the art. 

134. In my opinion, the asserted claims of the *381 patent recite a combination of prior 
art elements. As I explain below, this combination of known elements was obvious to try in light 
of prior work in the area of user interface design, including work on animation and “physics”’- 
based metaphors in user interfaces. (Indeed, as described above, this combination of known 
elements was in fact implemented in the LaunchTile and XNav systems to provide a user interface 
with animated “physics”’-based feedback for certain operations.) The idea of a user interface with 
animated “physics”-based feedback was not new in 2007. Each of these techniques had already 
been used alone or in combination to solve problems relating to movement and alignment. In my 
opinion, the combination of these elements is essentially all that is required to practice the asserted 


claims, as follows. 


-39- Case No. 11-cv-01846-LHK 


DECLARATION OF ANDRIES VAN DAM, PH.D. INSUPPORT OF SAMSUNG’S OPPOSITION TO 
APPLE’S MOTION FOR A PRELIMINARY INJUNCTION 





NY Dn OW HR WOW Ye 


Oo ow 


10 
11 
12 
13 
14 
15 
16 
Ly 
18 
19 
20 
21 
22 
23 
24 
23 
26 
os | 
28 


Case5:11-cv-01846-LHK Documenti168 Filed08/22/11 Page41 of 50 


135. For several decades, the concepts of using physics-based metaphors (for example, 
magnetism, gravity, or friction) in human computer interaction have been known and discussed 
widely in the field of user interface design and computer graphics. Many papers discus the use of 
“physics” in user interface design to make human-computer interactions seem more realistic to a 
user. These physics-based metaphors tap into a user’s pre-existing knowledge regarding the 
physical world, thereby making the user interface feel immediately familiar and comfortable. I 
note that Apple’s own publications from the early 1990s on user interface design discussed the 
utility of “physics” as a user interface metaphor. See, e.g., Thomas D. Erikson, Working With 
Interface Metaphors, in Brenda Laurel, ed., The Art of Human-Computer User Interface Design 
(Addison Wesley 1990), at 65-73. As this publication notes: 


[OJne particularly useful approach is to focus on the user problems you've 
identified and look for real-world events, objects, or institutions that embody 
some of the characteristics that users find difficult to understand. These make 
good candidates for new interface metaphors. For example, what are some real- 
world things that exhibit directionality? Rivers flow in one direction; TV images 
are transmitted from a broadcaster to a receiver; newspapers are mailed from a 
publisher to subscribers; forces like gravity and magnetism have direction. 

Id. at 70. 


Other papers from this period provide specific examples of “physics”-based metaphors, such as 
simple collision “physics” in which objects bump into and displace each other. See, for example, 
George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten 
van Dantzich, Data Mountain: Using Spatial Memory for Document Management (UIST 1998) at 
153-162. In this metaphor, multiple web pages displayed at once on a screen in a three- 
dimensional space could be dragged and collided to “transitively propagate displacement” to other 
pages. These collisions occurred in real-time so that the user “sees what state will result when the 
drag is terminated.” Because the displayed web page objects collide to displace each other, ““you 
cannot move a page and leave two pages in the same location,” meaning that “the pages never get 
fully obscured.” This approach is a direct application of simple collision “physics” to a user 
interface. 

136. As an additional example, computer-aided design (CAD) programs have used 


physics-based metaphors for attractive forces, such as gravity or magnetism, for many years in 
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user interfaces. CAD programs often include so-called “grid gravity” mechanisms for object 
alignment. In these systems, the user interface is often organized into a Cartesian grid; each 
element of the grid is imbued with attractive properties akin to a gravitational or magnetic force, 
such that when an object on the user interface comes within some small distance of a point on the 
grid, that object will “snap” to the grid. This system aids the user in aligning objects with the 
points or lines of the grid, or aligning edges of objects. These concepts are not limited to a 
Cartesian grid and can be used with vertices, edges, or other geometric features to align objects of 
arbitrary complexity, either with a grid or with each other. These snapping techniques have 
obvious utility for CAD programs and have therefore been an element of CAD programs for 
decades. See Mark Gross, Grids in Design and CAD, at 7 (ACADIA 1991, Los Angeles) (noting 
that “[g]rid snap behavior, or ‘grid gravity,’ is a feature in almost every drawing and drafting 
program’). Grid gravity mechanisms have obvious utility for many other user interfaces, in 
particular where it is advantageous to enable precise placement and alignment of user interface 
elements. 

137. Among the many publications discussing the advantages “physics”-based 
metaphors offer in general in user interface design is Benjamin Bederson & James D. Hollan, 
Pad++: A Zooming Graphical Interface for Exploring Alternate Interface Physics (Association of 
Computing Machinery, 1994). Bederson and Hollan teach several advantages of physics-based 
metaphors over other popular user interface metaphors. For example, when a UI metaphor is no 
longer useful or prevalent, as is common in the rapidly developing world of computer technology, 
a whole series of code based on that metaphor may need to be scrapped. For example, Bederson 
and Hollan note that “menu” and “desktop” metaphors frequently evolve in a manner that may 
cause prior code bases to be jettisoned. However, when the metaphor is “physics,” it is unlikely to 
ever have to be scrapped, as the laws of physics are ubiquitous and unchanging. Another known 
advantage to physics-based metaphors is the reduced learning curve. One cost that comes with 
any new UI metaphor “is that designers can no longer rely on users’ familiarity with the [older] 
metaphorical reference[s] and this has learnability consequences.” (p.25). However, these 


learnability problems are diminished when the metaphor is based on physics, as “empirical 
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knowability naturally follows from a physics perspective.” (p.25). A person of ordinary skill, 
having familiarity with publications such as Bederson and Hollan, would understand that users 
quickly grasp the design of interfaces that mimic attributes of physical systems, since these 
interfaces build on the users’ intuition and their experiences with the physical world. 

138. Assorted “physics” metaphors and their benefits for user interface design are 
discussed in Bay Wei Chang and David Ungar, Animation: From Cartoons To The User Interface, 
Sun Microsystems Laboratories Inc. Technical Report No. SMLI-TR-95-33 (March 1995). As an 
example, Chang and Ungar discuss techniques used in the Self user interface to replicate the 
“physics” of real-world interactions. These include collision “physics” metaphors in which an 
object “reacts to the contact with a small wiggling jolt” (p. 7 & Fig. 4), spring “physics” 
metaphors in which an object that is about to move “anticipates that move with small, quick 
contrary movement” similar to “the Coyote in the Road Runner cartoons, springing back onto its 
rear leg before dashing off after the Road Runner” (p. 9 & Fig. 7), acceleration and deceleration 
“physics” used on moving objects to begin motion slowly and end motion slowly, “giv[ing] a 
feeling of weight to the [object] and physicality to the movement” (p.10 & Fig. 8), and inertial 
“physics” that cause objects that cause “[o]bjects coming to a stop [to] wiggle at the end of their 
motion, as if reacting to a small spring at the end of their travel” (p.11 & Fig. 10). Each of these 
metaphors had long been known in the cartoon animation realm; Chang and Ungar merely 
recognize that these same principles of animated “physics” are useful as applied to user interface 
design. 

139. Apple’s expert, Dr. Balakrishnan, has observed the same advantages to the use of 
“physics” in the user interface in his publications. For example, Dr. Balakrishnan addressed this 
subject in Keepin’ It Real: Pushing the Desktop Metaphor with Physics, Piles and the Pen, 
Proceedings of the ACM (2006 Conference on Human Factors in Computing Systems (CHI) at p. 
1283-1292). In this paper, Dr. Balakrishnan appealed to the use of physical systems in user 
interfaces, and in particular took the model of a desktop that could have objects piled on top of 
each other, rather than filed in two-dimensional folders, and then modeled the movement of these 


objects using “a variety of interaction and visualization techniques” that are “influenced by 
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friction and mass, much like we would manipulate lightweight objects in the real world.” (Id. at 
Abstract (emphasis added).) The paper echoes earlier publications lauding the utility of “physics” 
metaphors in user interface design: 


Physics-based movement of objects is simulated with rigid body dynamics, 
collision detection, and frictional forces. When objects collide they bump against 
and displace one another in a physically realistic fashion. A simulated 
gravitational force keeps objects on the ground. The introduction of physics 
simulation to a desktop environment makes the desktop more lively, and offers 
increased degrees-of-freedom for potentially more expressiveness than a 
traditional GUI desktop where icons are kept axis- aligned and have little 
resemblance to their physical counterparts. We believe that this physical 
simulation has a positive and subtle effect on object placement and appearance. 
For example, if a few documents are casually tossed to a corner they will collide 
and begin to accumulate. Their messy appearance subtly affords an unorganized 
state, without the user having to explicitly specify it. 


Id. at 1286. 

140. In addition to physics-based metaphors, the use of animated transitions in user 
interface design has long been a well-known concept in the field, included animated translations. 
Chang and Ungar, cited above, discuss this history and these issues at length in Animation: From 
Cartoons To The User Interface, Sun Microsystems Laboratories Inc. Technical Report No. 
SMLI-TR-95-33 (March 1995). As Chang and Ungar explained, techniques drawn from cartoon 
animation replace sudden changes to a computer’s display with smooth animated transitions. 
These animated transitions are beneficial for users and offer both cognitive and affective benefits. 
Animation eliminates the need for a user to puzzle over how and why user interface elements 
changed in response to object movement or translation. 

141. A person of ordinary skill in the art would appreciate that and the combination of 
animation and “physics” metaphors is a predictable and in fact common combination, particularly 
for animated physics-based metaphors such as gravity, magnetism, spring forces, elastic forces, or 
others (see Chang and Ungar for a more complete list). Each of these techniques were known 
solutions to problems in user interface design. In the ‘381 patent, these known techniques are 
combined as part of a metaphor mimicking the movements of physical objects impacted by 


attractive forces (e.g., magnetism or gravity) or reactive forces (e.g., an elastic band or spring 
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contracting, or a ball bouncing off a solid surface). There is nothing novel or surprising about 
using any of these metaphors to signal and correct a user’s over-scrolling or under-scrolling 
operation. In fact, as described above and in Exhibits 2 - 3 hereto, prior art systems such as 
LaunchTile and XNav actually employed such a technique. 

142. It would be natural to one of skill in the art to employ a physics-based effect 
mimicking attractive or reactive forces when designing aspects of a touch-screen user interface. 
Such a mechanism offers known solutions to alignment problems for object edges, including 
misalignment due to a user’s over-scrolling or under-scrolling operation or other form of 
movement. A mechanism of this sort is useful not only for aligning documents or document 
portions that have been scrolled beyond the edge of the document currently being examined, but 
also for correcting alignment errors involving an electronic document or portions thereof. A 
person of ordinary skill in the art would immediately appreciate that these alignment mechanisms 
are useful to prevent wasteful use of screen space on a mobile device’s small screen; absent such a 
snap-back effect, the screen’s small display may be used inefficiently by displaying an area 
beyond the edge of a document being examined due to an imprecise scrolling operation. The use 
of an animated metaphor to re-align elements of the display to maximize the visible portion of the 
document being examined would have been natural to a person of ordinary skill in the art, 
including metaphors based on physical properties such as attractive forces and reactive forces. 

143. The techniques described above were well within the competence of a person of 
ordinary skill in 2007 (and for many years prior), on their own or in combination. I note that the 
‘381 patent itself appears to presume that a person of ordinary skill would know how to implement 
each of the individual techniques that the patent describes, such as document translation, 
animation, and implementing an animated physics-based metaphor for document translation (e.g., 
a “rubber band” metaphor mimicking an elastic force). For example, the patent does not contain 
source code or even high-level pseudo-code for implementing this functionality. Algorithms for 
implementing this functionality are not described in the specification or disclosed in the patent’s 
high-level flow diagrams. Instead, the patent’s high-level flow diagrams presume that techniques 


for implementing the functionality disclosed in those high-level flow diagram are already known 
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to persons of skill. In my opinion this is a fair assumption, because techniques for implementing 
the claimed functionally were already well-known in the art and within the grasp of a person of 
ordinary skill. 

144. Lunderstand from counsel that secondary considerations that could prove instructive 
on the issue of obviousness include commercial success, failure of others, and long felt but 
unresolved needs. I am not aware of any evidence that ties the success of any product, including 
any Apple product, to any particular claims of the ‘381 patent. Iam also unaware of any 
evidence that others in the field tried, but failed, to formulate the combination of prior art elements 
recited in the *381 patent’s claims. To the contrary, others in the field (for example, Dr. 
Bederson’s team on the LaunchTile and XNav projects) did in fact create systems that disclose 
each and every limitation recited in the ‘381 patent claims. Finally, with regard to long felt but 
unresolved needs, in my opinion there was no long felt need for the combination of known 
techniques described in the ‘381 patent claims. The problem of moving objects “beyond the 
edge” is as old as computer graphics, and many classical solutions exist (e.g., clipping, wrap- 
around, etc.). The specific problem of preventing a user from moving an electronic document 
beyond the edge of its window treated in the patent also has a number of obvious solutions. One 
is to prevent the document from moving beyond the edge by ignoring further requests for any such 
movement. This can be done with or without visual feedback indicating that these requests will 
be ignored. The patent describes one technique for visual feedback, which is to allow the user to 
temporarily have the illusion of scrolling or dragging the edge of the document beyond the edge of 
the window, but then immediately animating a “snap” back to a state where the document is not 
permitted to move beyond the edge. I am aware of no long-felt, unresolved need for this 
particular form of visual feedback. 

145. Should Apple take a position on the considerations discussed above, or present 
evidence relating to any of these topics, I reserve my right to supplement or amend my opinions 
and respond to Apple’s assertions as appropriate. 


IX. MATERIALITY OF THE PRIOR ART 
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146. I have been asked to consider whether the prior art references discussed in Exhibits 2 
- 5 are cumulative of art that was cited during prosecution of the ‘381 patent. 

147. I understand from counsel that a reference is not cumulative of other cited art if it 
includes limitations or combinations of limitations that were absent from the prior art considered 
by the Examiner. Iam also informed that a reference that includes a more complete combination 
of all the relevant features, even if those features were in front of the Examiner, is not cumulative. 

148. During prosecution of the *381 patent, the applicants represented to the patent office 
that four references were most closely related to the subject matter of the claims: (1) Zimmerman 
et al., U.S. Patent No. 6,690,387, (2) Kwatinetz et al., U.S. Patent No. 5,495,566, (3) Pallakoff, 
U.S. Patent App. Pub. 205/0012723, and (4) Miller, “PersonalJava Application Environment.” 
Applicants asserted that this art did not contain any disclosure for multiple limitations in the 
claims, including any disclosure of a method wherein the second direction is opposite the first 
direction (claim 10). See April 30, 2008 Supplemental Accelerated Examination Support 
Document at 9, 18. The patent examiner concluded that the cited prior art “does not teach in 
response to an edge of the electronic document being reached while translating the electronic 
document in the first direction while the object is still detected on or near the touch screen display: 
displaying an area beyond the edge of the document, and displaying a third portion of the 
electronic document, wherein the third portion, is smaller than the first portion; and in response to 
detecting that the object is no longer detected on or near the touch screen display, translating the 
electronic document in a second direction until the area beyond the edge of the electronic 
document is no longer displayed to display a fourth portion of the electronic document, wherein 
the fourth portion is different from the first portion.” October 29, 2008 Examiner’s Amendment 
at 4-5. 

149. In my opinion, the prior art references discussed in Exhibits 2 - 5 are not cumulative 
of the references that were presented to the Examiner during prosecution of the ‘381 patent. For 
example, the prior art references discussed in Exhibits 2 - 5 disclose limitations that the applicants 
represented were not present in the references that the applicants identified as most closely related 


to the subject matter of the claims, such as a method wherein translation in a second direction 1s 
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opposite the first direction (claim 10). See Exhibits 2-5. Likewise, the prior art references 
discussed in Exhibits 2 - 5 disclose a method of translation that involves the display of a first 
portion, second portion, third portion and fourth portion of an electronic document. J/d. Further, 
as explained in Exhibits 2 - 5, in my opinion the prior art references discussed in Exhibits 2 - 5 
would teach and inspire one of ordinary skill in the art to perform the limitations identified by the 
patent examiner as missing from the cited art. Compare Exhibits 2 - 5 to October 29, 2008 
Examiner’s Amendment at 4-5. In light of the fact that the examiner indicated that these 
limitations were the only ones the examiner said were missing from the art, the prior art references 
discussed in Exhibits 2 - 5, would have, alone or in combination with one or more of the cited 
prior art references, rendered the claims unpatentable. 

150. Moreover, it is my opinion that the prior art references discussed in Exhibits 2 - 5 
are not cumulative of art cited during the reexamination of the ‘381 patent. Specifically, the 
examiner stated that the claims were patentable over the prior art cited during reexamination 
because “[t]he prior art does not teach or fairly address the invention as recited in independent 
claims 1, 19, and 20 of U.S. Patent No. 7,469,381 82. Specifically, the prior art references of 
Glimpse, Inside Out, Robbins, and Zimmerman fail to disclose ‘in response to detecting that the 
object is no longer on or near the touch screen display, translating the electronic document in a 
second direction until the area beyond the edge of the electronic document is no longer displayed 
to display a fourth portion of the electronic document, wherein the fourth portion is different from 
the first portion’ as recited in claims 1, 19, and 20.” ‘381 reexamination file history, Notice of 
Intent to Issue Ex Parte Reexamination Certificate (January 13, 2001) at 2. As already discussed 
above, the prior art references discussed in Exhibits 2 - 5 expressly disclose these limitations. 
Therefore, in my opinion, the prior art references discussed in Exhibits 2 - 5 is not cumulative of 
the art cited during reexamination of the ‘381 patent. 

X. OTHER COMMENTS 
151. The opinions expressed in this declaration are my preliminary opinions based on my 
review to date of the evidence produced at this early stage of the case. My opinions are subject to 


change based on additional opinions that Apple’s experts may present and information I may 
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receive in the future. I reserve my right to amend or update my opinions as appropriate in 
response to any future developments regarding claim construction, including any court order on 
claim construction. With this in mind, based on the analysis I have conducted and for the reasons 
set forth above, I have preliminarily reached the conclusions and opinions in this declaration. 

152. At a hearing or at trial I may use as exhibits various documents produced in this case 
that refer or relate to the matters discussed in this report. I have not yet selected the particular 
exhibits that might be used. I may also rely on visual aids and may rely on analogies concerning 
elements of the ‘381 patent, the accused products, the prior art referenced in this report, or any 
related technologies. In addition, I may create or assist in the creation of certain demonstrative 
evidence to assist me in testifying, and I reserve the right to do so, such as demonstrations of 


devices and software to further support the positions in this report. 
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I declare under penalty of * em that the foregoing is true and correct. Executed in 


Providence, Rhode Island on , 2011. 


- on 


Andries Van Dam 
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